Variance-Reduced Stochastic Learning by Networked Agents under Random Reshuffling

This work develops a fully decentralized variance-reduced learning algorithm foron-device intelligence where nodes store and process the data locally and are onlyallowed to communicate with their immediate neighbors. In the proposed algo-rithm, there is no need for a central or master unit while the objective is to enablethe dispersed nodes to learn the exact global model despite their limited localizedinteractions. The resulting algorithm is shown to have low memory requirement,guaranteed linear convergence, robustness to failure of links or nodes, scalabilityto the network size, and privacy-preserving properties. Moreover, the decentral-ized nature of the solution makes large-scale machine learning problems moretractable and also scalable since data is stored and processed locally at the nodes.